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Abstract 

This paper develops a dissipativity theory for dynamical systems governed by linear Ito stochastic differential 
£NJ . equations driven by random noise with an uncertain drift. The deviation of the noise from a standard Wiener 

process in the nominal model is quantified by relative entropy. The paper discusses a dissipation inequality for the 
noise relative entropy supply. The problem of minimizing the supply required to drive the system between given 
Gaussian state distributions over a specified time horizon is considered. This problem, known in the literature 
as the Schrodinger bridge, was treated previously in the context of reciprocal processes. The paper obtains a 
closed-form smooth solution to a Hamilton-Jacobi equation for the minimum required relative entropy supply 
. by using nonlinear algebraic techniques. 

I. Introduction 

i — ,! We consider a dynamical system whose state is a diffusion process governed by a linear Ito stochastic 
differential equation (SDE) driven by a random noise. The noise is generated from a standard Wiener 
process by another SDE with an uncertain drift. The case where the drift vanishes and the noise replicates 
the Wiener process, represents the nominal scenario. A nonzero drift in the noise SDE can be interpreted 
as the strategy of a hypothetical player who uses the past history of the system state in order to move 
its probability density function (PDF) away from the nominal invariant state PDF. The deviation of the 

'— 1 actual noise distribution from the Wiener measure can be quantified by the Kullback-Leibler relative 
entropy [5]. As a measure of uncertainty in the noise distribution, the relative entropy is often utilized 
>• \ in the robust control of stochastic systems [4], [7], [15], [17]. 

The noise relative entropy over a bounded time interval can be regarded as a stochastic analogue of 
the supply which is a fundamental concept in the theory of deterministic dissipative systems [19]. This 
analogy leads to a dissipation inequality which links the noise relative entropy supply with the increment 



in the relative entropy of the state PDF of the system with respect to the nominal invariant state PDF. 
The state relative entropy, therefore, plays the role of a storage function. The relative entropy dissipation 
inequality is related to Jarzynski's equality [11] for the Helmholtz free energy in open dynamical systems. 
This non-equilibrium thermodynamics viewpoint, where the noise results from interaction of the system 
with its surroundings (via mechanical work and heat transfer), motivates a stochastic dissipativity theory 
in the form of a variational problem involving entropy. Such problems are more complex than their 
deterministic counterparts since they deal with probability measures (or PDFs) on signal spaces, rather 
than the signals themselves. 

We are mainly concerned with computing the minimum noise relative entropy supply required to drive 
the system between given initial and terminal state PDFs over a specified time horizon. The state PDF 
transition problem, known as the Schrodinger bridge, was treated previously in a context of reciprocal 
processes (Markov random fields on the time axis) [1], [3], [6], [13]. This problem was also studied for 
quantum systems [2], using the formalism of stochastic mechanics [14]. The solution of the Schrodinger 
bridge problem is related to two coupled integral equations [13, Definition 2.3 on p. 26] and is not 
available in closed form for a general diffusion model. 

We consider the state PDF transition problem with Gaussian initial and terminal state PDFs and under- 
take a different, somewhat more algebraic, approach. Using Markovization and stochastic linearization of 
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the noise strategy as entropy-decreasing operations, we establish a mean-covariance separation principle 
which splits the minimum required noise relative entropy supply into two independent terms associated 
with the mean and covariance matrix of the system state. While the mean part is calculated using 
standard linear quadratic optimization, the covariance part (which is a function of matrices) satisfies a 
Hamilton-Jacobi equation (HJE) complicated by inherent noncommutativity. 

This partial differential equation (PDE) has a quadratic Hamiltonian on its right-hand side (with the 
quadraticity coming from the diffusion part of the system dynamics) and involves a certain boundary 
condition. The bilinear "interaction" of solutions of this PDE (which, in the quadratic case, replaces 
the superposition principle) allows them to be generated in a quasi-additive way. Unlike infinitesimal 
perturbation techniques based on asymptotic expansions in the small noise limit, our approach provides 
& finite correction scheme which allows a closed-form smooth solution to be found for the HJE. 

The correction scheme also employs an ansatz class of "trace-analytic" functions of matrices and 
a matrix version of the separation of variables which not only copes with the nonlinearity but also 
essentially "scalarizes" the covariance HJE, thus overcoming the noncommutativity issues. These 
nonlinear algebraic techniques may therefore be of interest in their own right from the viewpoint of 
nonlinear PDEs and holomorphic functional calculus. 

II. Class of systems being considered 

We consider a dynamical system whose state X := (X t )t^o is a diffusion process in M. n governed by 
an Ito SDE 

dX t = f(X t )dt + BdW t , f(x):=fx + Ax, (1) 

driven by an Revalued random noise W := (W t ) t>0 . Here, // eR", A G R nxn , B G R nxm , with A 
Hurwitz and rank5 = n ^ m, so that the diffusion matrix 

D := BB T (2) 

is positive definite. The noise W is an Ito process interpreted as an external random noise which 
originates from interaction of the system with its environment and is generated by another SDE 

dW t = h t dt + dW t , (3) 

with an uncertain drift h := (h t ) t ^ . Here, h is a random process with values in R m , adapted to the 
natural filtration (J r t)t^o of X, where JF t is the cr-subalgebra of events induced by the history X\o )t ] 
of X on the time interval [0,t]. Also, W := (Wt)t^o is an m-dimensional standard Wiener process, 
independent of X . Substituting ([3]) into (OQ) yields 

dX t = (f(X t ) + Bh t )dt + BdW t . (4) 

We assume that E J*\h s \ 2 ds < +oo for all t > 0. Together with D y 0, the local mean square 
integrability of h ensures the absolute continuity of the system state X t . The case h = (where 
W = W) represents the nominal scenario of the system-environment interaction. A nonzero drift h t is 
interpreted as the strategy of a hypothetical player who uses the past history X\qa of the system state 
X t to move the state PDF p t away from the nominal invariant state PDF 

p.(x) = (27r)-"/ 2 (detn,)- 1 / 2 exp(-||x-a,||^ 1 /2), (5) 

which the system would have in the nominal case. The nominal invariant state distribution is Gaussian, 
Af(a*,U*), with mean a* and covariance matrix II* given by 

r+oo 

a, = -A"V, n*= / e M De AFt dt, (6) 
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where II* is the infinite-horizon controllability Gramian of the pair (A, B) satisfying the algebraic 
Lyapunov equation 

ATI* + Tl*A T + D = 0. (7) 
The deviation of the actual noise distribution from the Wiener measure is quantified by 

Et := D(P t ||P*) - D(P ||P*) = ~ j E(\h s \ 2 )ds. (8) 

Here, Girsanov's theorem [9] is used; D(M\\N) := E M ln(dM/diV) is the Kullback-Leibler relative 
entropy [5] of a probability measure M with respect to another probability measure N (under the 
assumption of absolute continuity M <C N); and P t and P£ are the restrictions of the true and nominal 
probability measures P and P* to the a-algebra a(X , W[ 0jt ]). The expectation E in © is over P under 
which the noise W, governed by ©, becomes a standard Wiener process if and only if h = 0. The noise 
relative entropy E t over the time interval [0, t] from © can be regarded as a stochastic counterpart of 
the supply in the theory of deterministic dissipative systems [19]. 

In the nominal case h = 0, the state X of the system is a homogeneous Markov diffusion process 
and the state PDF p t satisfies the Fokker-Planck-Kolmogorov equation (FPKE) 

d m = tffa), (9) 

where 

C\p) := div 2 (Dp)/2 - div(/p). (10) 

Here, for any twice continuously differentiable function G := {Gij)i^i,j^n '■ ^ n §>„, with § n the 
space of real symmetric matrices of order n, the maps divG : M n — > W 1 and div 2 G : W 1 — > R are 
defined by divG := {YJj=i ^jGij)i<i<n and div 2 G := divdivG = ^ 1<iii<n VfVjGy, where V« := d x . 
is the partial derivative with respect to the ith Cartesian coordinate in IR n . The operator C) in (flOl) is the 
formal adjoint of the infinitesimal generator C of X in the nominal case. The action of £ on a twice 
continuously differentiable test function cp : M. n — > R with bounded support is described by 

£( V 9) = / T V^ + Tr( J D V 9' / )/2, (11) 

where (•)" is the Hessian matrix. Since A is Hurwitz, the system is ergodic under the nominal noise 
W = W and the PDF © is a steady-state solution of the FPKE ©: £ f (j?*) = 0. The controllability of 
(A, B) (which follows from D y 0) is equivalent to II* y 0, and is also equivalent to the nonsingularity 
of the finite-horizon controllability Gramian 

1 e As De ATs ds = II„ - e At TL*e ATt (12) 



for any t > 0. We define two semigroups of affine transformations (M t )t^o and (C t )t^o by 

M t (a) := e At a + A~ 1 (e At - I n )/i = a, + e At (a - a,), (13) 
C t (S) := e At Ze ATt + T t = II* + e A *(E - n*)e AT *. (14) 

These semigroups act on R n and the set of real positive semi-definite symmetric matrices of order 
n and describe the nominal evolution of the state mean and covariance matrix 

a t := EX t , U t := cov(X t ). (15) 

The infinitesimal generators of the semigroups are given by 

M(a)=j2 + A aj = ATj + TjA t + D. (16) 
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In general (when h ^ 0), the linearity of the SDE © allows the dynamics of <fT5T ) to be described by 

a t = M(a t ) + U t = C(U t ) + BK t U t + U t KjB T (17) 

in terms of the moments 

Pf.= Eh t , K t :^cov(h t ,X t )U-\ (18) 

III. MARKOVIZATION AND STATE PDF DYNAMICS 
For any t ^ 0, we define a function h t : M n — > M. m associated with the noise strategy h by 

h t (x) :=E(h t \X t = x). (19) 

In particular, if h t is a deterministic function of t and the current state X t , then 

h t = h t (X t ), t^O. (20) 

The noise strategies h, satisfying (|20l ) with probability one, are said to be Markov with respect to the 
state of the system. 

Proposition 1: Suppose that the state PDF p t (x) of the system, governed by © is continuously 
differentiable in t > and twice continuously differentiable in x E M n . Then it satisfies the FPKE 

d tPt = dw 2 {Dp t )/2 - div((/ + Bh t )p t ) = C\p t ) - div(Bh t p t ), (21) 

where the operator & is defined by (flOl) . 

Proof: In view of the smoothness of p t and the identity E(/(X t ) + Bh t \X t = x) = f(x) + Bh t (x) 
which follows from (TT91 ). the PDE (I2TI) is obtained from the weak formulation of the FPKE for Ito 
processes in [13, Eqs. (0.1 1)— (0. 13) on p. 21]. ■ 

The PDE ([2T|) governs the PDF of a Markov diffusion process £ := (£t)t>o generated by the SDE 
d& = (/(&) + Bh t (£ t ))dt + BdW t . If £o and X are identically distributed with the state PDF p , then 
£t and X t share the common PDF p t for any £ > 0. The passage h t i-> h t (X t ) from an arbitrary noise 
strategy /i to the Markov strategy, defined by (fT9l . is referred to as the Markovization of ft, and denoted 
by M. Although M preserves the state PDFs p t of the system, the multi-time probability distributions 
of X are, in general, modified. They all remain unchanged under the Markovization M if and only if 
h is Markov. Since such strategies are invariant under M, the Markovization is idempotent: M 2 = M. 

Theorem 1: The Markovization M of a noise strategy h does not increase the noise relative entropy 
supply ©: 

Et>\J^ E(|77 t (X t )| 2 )dt, (22) 

where h t is given by (fT9l) . This inequality is an equality if and only if h is Markov in the sense of (l20l) . 

Proof: The standard properties of iterated conditional expectations, strict convexity of the squared 
Euclidean norm | • | 2 and Jensen's inequality imply that E(|/i 4 | 2 ) = EE(|ftt| 2 |X t ) ^ E(|/i t (X t )| 2 ), where 
the inequality becomes an equality if and only if (|2Q|) holds with probability one. Integration over 
t E [0,T] yields (1221) whose right-hand side is completely specified by the functions h t and p t . Since 
the state PDFs remain unchanged under the Markovization of the noise strategy, then (l22l) holds as an 
equality if and only if h is Markov. ■ 
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IV. Relative entropy dissipation inequality 
We will now consider the state relative entropy defined by 

R t :=Elnq t {X t ) = (p t ,lnqt), (23) 

where (a, b) := f Rn a(x) T b(x)dx is the inner product of functions a, b : M n — > W (provided the integral 
exists, as is the case, for example, when a, b are square integrable), and 

ft ■= Pt/p* (24) 

is the true-to-nominal state PDF ratio. Note the difference between R t and the noise relative entropy E t 
defined in ([8]); see [6, Eq. (3.12) & Remark on p. 321]. In the nominal case, R t is non-increasing in t, 
which represents the Second Law of Thermodynamics for homogeneous Markov processes as models 
of isolated systems [5]. A nonzero h makes © an open system and R t is no longer monotonic. 
Theorem 2: For any T > 0, the increment R T — R of the state relative entropy (1231 satisfies 

i r T 

R T -R ^E T --J (p t ,\h t -5 T V 'In ft| 2 )dt, (25) 

where h t and q t are defined by (fT9l) and ((24)) . This inequality is an equality if and only if h is a Markov 
noise strategy. 

Proof: Differentiation of the right-hand side of (1231 gives 

d t Rt = (d t pt, In g t ) + (puP^dtPt) = {£ ] {Pt) - div(gh t p t ), In ft). (26) 

Here, use is made of (1211) and the identities d t In q t = Pfd t p t and (1, d t p t ) = which follow from (1241) 
and (l,pt) = 1. Integration by parts reduces (l26l) to 

d t R t = (Pt, CQnqt)) + (gh tP t,V In q t ). (27) 

In view of Fleming's logarithmic transformation [8] (see, also, [2, Eq. (81) on p. 201]), the operator C, 
defined by (flTT) . acts on the logarithm of a twice continuously differentiable function ip : IR n — > (0, +oo) 
as C(lnip) = C(ip)/ip — \\V \mjj\\ 2 D /2. Application of this relation to ip := q t from ([241) represents the 
first inner product in (l27l in the form 

(p t ,£(lnft)) = (pt,C(qt)/q t ~ ||Vlnft||y2) 

= (p*,£(q t )) - (p t , ||Vlnft||^)/2 = -(p t , ||Vlnft||^)/2, (28) 

where (p*,C(q t )) = (&(p*),qt) = 0. By substituting ([28]) into ((271), it follows that 

d t R t = (pt,h^B T V\nqt-\\V\nq t \\ 2 D /2} = (p u \h t \ 2 - \h t - B T V In ft| 2 )/2, (29) 

where the square is completed using ©. Integration of both parts of (l29l in t over [0, T] yields 

Rt-Ro + 1 [ (Pt, \ht ~ B T V\nq t \ 2 )dt = \ [ E(|^(X t )| 2 )dt < E T , (30) 
^ Jo ^ Jo 

where the inequality proves ([221) . Now, ([251 follows from (l30l) . The claim that (l25l) holds as an equality 
if and only if h is Markov, follows from the second part of Theorem \T\ ■ 
The relation (l25l) can be regarded as a dissipation inequality [19, pp. 327, 348], with the state relative 
entropy (l23l playing the role of a storage function. Thus, the absolute continuity of the state distribution 
of the system can only be destroyed within the finite time T using an infinite noise relative entropy 
supply Et- Note also that ([251 . which becomes an equality for Markov noise strategies, can be thought 
of as an analogue of Jarzynski's equality from nonequilibrium thermodynamics [11]. 
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V. Gaussian state PDF transition 



We will now consider the problem of driving the system © from a Gaussian initial state PDF p := a 
to a Gaussian terminal state PDF px := 9 at a specified time T > so as to minimize the supply ([8]) 
over the interval [0, T]: 

J T (a, 9) := inf {£ T : p = a, p T = 9}. (31) 

Here, 

a ~ A/"(a ,n ), 9 ~ Af^ctT, riy), (32) 

with n ,n T >- 0. The required supply (|3TT) vanishes if and only if 9 is nominally reachable from 
a in time T in the sense that 9 = e ctT (a). Here, e £tT is the linear integral operator (with a 
Markov transition kernel) which relates the terminal state PDF p T of the system at time T with the 
initial state PDF p under the nominal FPKE Q. In particular, Jt(p*,P*) = and, more generally, 
J T (A/"(a, S), Af(M T (a), Cr(£))) = 0. In CD}, the intermediate state PDFs p t , with < t < T, are not 
required to be Gaussian. Nevertheless, we can restrict attention to noise strategies which are not only 
Markov, but are also affine with respect to the state of the system, thus making the intermediate state 
PDFs also Gaussian; see Fig. [TJ 




Fig. 1. The cylinder represents the set G x [0, T] whose base G is formed by nonsingular Gaussian PDFs in K™. The minimization 
of the noise relative entropy supply Et in driving the system state PDF between a, 9 G G in time T can be restricted to affine Markov 
noise strategies without affecting the minimum. Such strategies generate paths of Gaussian state PDFs from a to 9 which are entirely 
contained by the cylinder. 



VI. MEAN-COVARIANCE SEPARATION PRINCIPLE 

For any T > 0, we define a function St '■ — > M + , with P„ the set of real positive definite symmetric 
matrices of order n, by 

Sr(S, 6) := inf / Tr(K t n t K?)dt. (33) 
Jo 

Here, the minimization is over IR mxn -valued functions K t from (fT8b such that the state covariance 
matrix Yi t , governed by the second of the ordinary differential equations (ODEs) (fT7l) and initialized at 
n := E, satisfies the terminal condition Ht = ©• Since Ct(E) is reachable from £ in time T with 
K t = by the action of the nominal state covariance semigroup ([141) . then 

S , T (E,C7 T (S)) = 0. (34) 
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Theorem 3: The minimum required noise relative entropy supply in (l3"TT)-(l3~2l) can be computed as 



J T (a,9) = (||a r - M T (a )\\ 2 r ^ + S T (U ,U T ))/2, (35) 

where (fT2~l) . ([T3l , (1331) are used. The optimal noise strategy is an affine function of the current state of 
the system: 

h t = (3 t + K t (X t -a t ), (36) 



where K t is a function delivering the minimum in (|33l) . and 

A := B T e AT{T - t) T- 1 (a T - M T (a Q )), ^ t ^ T, (37) 
Proof: It follows from (fTVl) that the state mean a t is completely specified by the initial condition 
a and the function t from (fT8"l) . Similarly, the state co variance matrix Il 4 is completely specified by 
il and the function K t . The relative entropy supply ([8]) affords the lower bound: 

2E T = [ E(\h t \ 2 )dt^ / (\p t \ 2 + Tr(K t Il t K?))dt. (38) 
Jo Jo 

Indeed, by [10, Theorem 7.7.7 on p. 473], the Schur complement of the block cov(X t ) 
in the joint covariance matrix of X t and h t is positive semi-definite. Hence, cav (ht) ^ 
cov(h t ,X t )(cov(X t )y 1 cov(X t ,ht) = K t n t Kj in view of (QJ) and (QJ), and E(|^ £ | 2 ) = \(3 t \ 2 + 
Trcov(/i t ) ^ \f3 t \ 2 + Ti(K t U t Kj-), which implies (|38l) . This inequality becomes an equality if and only 
if h t is related to X t by the affine map (1361) with probability one. In view of (IT3T ) and (fT8l) , the passage 
from the original noise strategy h to the right-hand side of (l36l) describes a stochastic linearization of /t. 
We denote this operation by L. By construction, it is idempotent: L 2 = L. The stochastic linearization 
L yields an affine noise strategy which is not only Markov in the sense of (|20l) . but also preserves 
the first two moments of X. Under the affine noise strategy, the state process X is Gaussian, provided 
that the initial state X is Gaussian. Thus, if h is a noise strategy which drives the state PDF of the 
system from a to 9, described by (l32l) . in time T, then h := L(/i) is an affine Markov strategy under 
which X has the same mean and covariance matrix as it does under h. By the latter property, h also 
drives the state PDF of the system to the Gaussian PDF 9, but supplying the same or smaller noise 
relative entropy to the system over [0,T] in view of (|38T ). Therefore, the minimization of E T can be 
reduced, without affecting the minimum value, to a minimization over affine Markov noise strategies 
(1361). Hence, recalling ([32]) and ([33]), 



2 J T (a, 9) = inf [ (\(3 t \ 2 + Tr(K t U t K^))dt = S T (U , U T ) + inf f |/3 £ | 2 dt. (39) 
Jo Jo 

Here, the first infimum is over the functions (3 t and K t from (fT8l) such that the state mean a t and 
covariance matrix U t , governed by the ODEs (flTl) with initial conditions a and il , satisfy the terminal 
conditions specified by olt and 11^, whereas the second infimum is only concerned with t . By the 
first of the ODEs (flTT ), the boundary conditions on the state mean are equivalent to e A ( T ~^B(3 t dt = 
«t-Mt(qo), where (TT3l) is used. The second infimum in ([39l is found by solving the linearly constrained 
quadratic optimization problem and achieved at the function (5 t from ([37]) . with min J Q T \/3 t \ 2 dt = \\ax — 
M r (a )|| 2 i, which yields <[35])- ■ 

1 T 

The representation (1351 ) splits J T into two independently computed terms which are associated with 
the mean and the covariance matrix of the system state. This decoupling is similar to the separation 
principle of Linear Quadratic Gaussian control [12]. 
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VII. COVARIANCE HAMILTON-JACOBI EQUATION 

We will now consider the "covariance" part St of the minimum required supply J T from (1351) whose 
"mean" part is already computed in Theorem [3] 

Lemma 1: Suppose that £V(E, 0) from (l33l) is smooth with respect to T > and E >- 0. Then it 
satisfies the HJE 

d T S T = F(E, «9 S S T ), F(E, $) := Tr((C(E) - D$£)$), (40) 

where © and (fl6l) are used. 

Proof: From (fl7l) and (1331) . it follows that, under the assumption of smoothness, St satisfies a 
Hamilton-Jacobi-Bellman equation 

d T S T = Tr(C(E)$ T ) + min (Tr(ATEA: T ) + Tr((5ATE + EAT T B T )$ r )), (41) 

where $ T := d^S T - Since If Elf T + KY^&B + _B T $Efr T = (AT + B T $)E(K T + $B) - £ T $£$£ and 
E >- 0, the minimum in (|4TT) is only achieved at K = —B T $ T and is equal to — Tt(D$tE<Ev), which 
implies (|40l . ■ 



VIII. Correction scheme 

The Hamiltonian F in (l40l) is quadratic in its second argument. Hence, if St satisfies the covariance 
HJE, then 

S T :=S T + S T (42) 

is also a solution of (140b if and only if St, playing the role of a correcting function, satisfies a modified 
HJE 

d T S T = -F(£,<9 S S T ). (43) 

Here, F is obtained by correcting the Hamiltonian F by a term arising from the bilinear "interaction" 
between S T and St'- 

F(E, $) := F(E, $) - 2Tr( J D(a s 5 T )E<l>) = Tr((C(E) - £)$E)$), (44) 

with 

C(E) := (A - DdxS T )Z + E(A - D^s^t) 7 + D (45) 

obtained by modifying the matrix A in the infinitesimal generator C from ([Tot . The operator C depends 
parametrically on T and E through St- 

IX. Starting solution 



Ignoring, for the moment, the boundary condition (|34l) . we will find a particular solution St of the 
covariance HJE (l40l) as a starting point for the correction scheme (|42l) - (|45l) . Note that application of 
the Hamiltonian F from (gO]) to the function $ := <9 S lndet E = E _1 yields a constant: F(E, E _1 ) = 
Tr((AE + EA T +D-L>E- 1 E)E- 1 ) = 2TrA Furthermore, the class of those smooth functions S T (Z, ©), 
which are affine with respect to E, is closed under 8t and the action of the Hamiltonian St ^ 
F(E, B^St)- Finally, solutions of the quadratic HJE (|40l) do not obey the superposition principle, and the 
bilinear interaction of In det E and an affine function S T is described by Tr(DE~ 1 E9sS'r) = Tr(Dd^ST) 
which is also constant in E. These observations suggest looking for a particular solution of (l40l) in the 
class of functions 

S T (E,0) = lndet E + Tr(ES r (6)) + T r (9). (46) 

Here, H T (0) and T T (0) are smooth functions of the time horizon T > and the terminal state 
covariance matrix 6 with values in S n and R, respectively, with det St (6) ^ 0. 



8 



Lemma 2: The following function is a particular solution of the covariance HJE (|40b in the class 
(061): 

S r (£) := lndet U T {T) + TW T (£). (47) 
Here, the map U T : P n -> P n is defined using (fl2b . ([141 as 

c/ T (s) := r T 1/2 c T (s)r r 1/2 - i n . (48) 

Proof: By differentiating the ansatz (146b . it follows that 

dfSf = Tr(£<9 T E T ) + «9 T T T , <9 S S T = S" 1 + H T . (49) 

In view of (TT6b . substitution of <9eSt into (|40b yields 

F(E, <9 T £r) = Tr((AS + SA T + D - Z^ST 1 + H T )£)<9 E S T ) 
= Tr((AS + SA T - DS T S)(S" 1 + Hr)) 

= Tr((S T A + A T E T - E T DE T )Z) + Tr(2A - DSr). (50) 

By equating OtSt from (149b with the right-hand side of (|50b . it follows that the ansatz function St in 
(|46b is a solution of (|40b if and only if E T and T T satisfy the ODEs 

d T E T = E T A + A T E T - E T DE T , (51) 
d T T T = Tr(2A - DE T ). (52) 

Multiplication of the right-hand side of (15Tb by S^ 1 yields a matrix whose trace coincides with the 
right-hand side of ((52b . Hence, <9 r ln | detS T | = Tr(H T " 1 #r5r) = 8tTt, and 

= In | det 5r| + T — In | det H |. (53) 

Left and right multiplication of both parts of (ISTb by E T X yields a differential Lyapunov equation 

d T {~ T l ) = -S T 1 (9 r S r )S r 1 = D — AE^ - S T M T (54) 

for E T l with a unique equilibrium point E T = — IL7 1 ; cf. ([7]). The general solution of (l54b is expressed 
via (fL2b as 

S r = e ATr (H 1 + r T )-V T (55) 
Setting T := In I det S | in (l53b and S — ► 00 in (l55b gives 

E T = e ATT r^ 1 e AT , T T = 2TTtA - In det T T . (56) 



Substitution of d56j) into d46j) yields the particular solution of (gOJ described by (147])— (1481) . ■ 

X. Trace-analytic correction 

Since St — > —00 as £ — >■ 0, regardless of the choice of E T and T r , the class (|46b can not provide 
a_ nonnegative solution to the HJE (l40b . We will therefore correct St from (l47b by adding a function 
St := £r(£, 0) such that (|42]) is a nonnegative solution of (140b satisfying the boundary condition (134b . 
Substitution of <9 s St from (|49b into the correction scheme (|4"3b-(l4"5b yields 

C(S) 



&tSt = Tr(( (A - DH r )£ + E(A - DE T ) L - D -D(d E S T )^)d s S T ). (57) 

To find the correcting function St, as a solution to (l57b such that SV in (l42b is nonnegative and satisfies 
(l34b . we will employ yet another ansatz: 

S T := Trx(^T) + Pt, ■= V t Wt- (58) 
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Here, x is a nonconstant function of a complex variable, analytic in a neighbourhood of M + and real- 
valued on M. + . Also, ^y(O) and Pt(@) are smooth functions of the time horizon T and the terminal 
state covariance matrix with values in P n and E, respectively. The matrix ^t, which specifies the 
linear operator £ i-> fi T in (|58l ), enters the "trace-analytic" function Trx(fi T ) only through since 

Tr X (fi r ) = Ttx&t^tXVt) = Tr X (^S), (59) 
where we use the invariance of the trace under similarity transformations and their commutativity 
with analytic functions of matrices. To find suitable x , *ff T , p T , we compute the derivatives S = 
Tr((\i/\P _1 + + p and d s S = \&x'(fi)iJ> b Y usm S Lemma Hand its corollary (|A2)) - (|A3t 

in Appendix A, with the subscript T omitted for brevity, and ( ) := d T - Substituting the derivatives into 
(1571) yields 

Tr((^^" 1 + + p =Tr((C(S) - D^ X '(^)^)^x'(^) 

- Tr(*D*x'(£Wn + X'(m)), (60) 

where use is made of = from (1581) and the commutativity of x'(fy an ^ ^- Regrouping the 

terms of (|60l ) yields 

2 

^ Tr(G fcX fc(^T)) + d T p T = (61) 
fc=i 

for all T > and S, 6 >- 0. Here, 

G 1 (T, 0) := * T -D*t, (62) 
G 2 (T, 0) := (9 T ^ T )^ T 1 + ^-^t^t - ^ T (A - DH T )^ T 1 - ^ T X (A - DS T ) T ^ T (63) 

are § n -valued functions, with G%(T, 0) >- 0. Also, 

:=X / (w)(l + wx'(w)), X2M :=wx'(w) (64) 

are functions of a complex variable w, which inherit from % the analyticity in a neighbourhood of and 
real-valuedness on R + . Since Xi{ w ) = X2(w)(l + X2 (uj))/u}, then xi or X2 is not constant. For any 
given (T, 0), the map £ i-> fiy in (1581) is a bijection of P n , and hence, (loTI) is equivalent to 

2 

Tr(G fc (T, e)xfe(fi)) + «9 TPT = (65) 

fc=i 

for all T > and 0, >- 0. Application of the separation-of- variables principle of Lemma [5] from 
Appendix B to (1631 ) yields the existence of constants A,rGl such that the function x , which generates 
Xx, X2 in (l64l) . satisfies the ODE 

X V)(l + ux'W) + AwxV) + = 0, (66) 
and Gi and G 2 in (|62l and (|63l and p r from (|58l ) satisfy 

(^t^t)^ 1 + ^^t^t - Vt(A - DEt)^ 1 - ^t\ a - DZ t ) t Vt = X^tD^t, (67) 

d T pr = rTr(D^). (68) 

The PDEs (1671 ) and (1681 ) are solved in the lemma below and the result is then combined with the ODE 

Lemma 3: The following function is a solution of the modified HJE (1571) in the class (1581 ): 

SHE, 0) = Tr X (U T (Z)V T (e)) + pr{Q), (69) 
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where U T is given by (|48l ) and \ satisfies the ODE (1661 ). Here, for any A ^ 0, the map V T : P n — > F n 
is associated with (TT2l) by 

k t (6) : = r^UTte)- 1 !^ 172 , u T (e) := at- 1 + u(e), (70) 

where 0(6) is a § n -valued function of 6 only which satisfies 

U(&) > -An; 1 for A > 0, U(&) y for A = 0, (71) 
with n* given by ©. Also, 

PT (0)-p(0)-| ^(u^-ij-i) forA = ' (72) 

where p(0) is a M-valued function of only. 

Proof: Since vp- 1 ^- 2 + if- 2 ^- 1 = v[/- 2 (vl/ 2 )-vl/- 2 = _(\j/- 2 )- ? left and right multiplication of 
both sides of (1671) by yields a differential Lyapunov equation 

d T (^ 2 ) + DE T )^ T 2 + ^ 2 (A - DE T ) T + XD = (73) 

with respect to \I/y 2 . Its solution is expressed in terms of the fundamental solution of the ODE 

<9 T £ T = -(A - D~ T )fr = -(A - L>e ATr r-V T )£ T , (74) 

where Et is given by (l56l) . The relation t t = e At De ATf implies that the general solution of (1741) is 
£ T = e~ AT T T (, where £ G M n is an arbitrary constant vector. Hence, the solution of (1731) can be found 
in the form 

m~ 2 = e - AT T T U T T T e- ATT (75) 

by the variation of constants method, where 15 T (Q) is a S„-valued function of T, 6. Substitution of 
(1751) into (E3> yields 

d T U T = -ATy^D^r^ 1 = A9 T (T^ 1 ), (76) 



whence U T in (1701) is obtained by integration. Now, the right-hand side of (1751) is positive definite for 
all T > if and only if so is U T (Q). In view of dVOj), the condition U T (@) >- for any T > is 
equivalent to A ^ and (fTTT) . Indeed, 1) the controllability Gramian in (fT2"l) satisfies -< IY -< IT* for 
all T > 0, and strictly -<-monotonically approaches and II* as T tends to and +oo, respectively; 
2) (-) -1 is a decreasing operator on the set P n with respect to the partial ordering -<. Therefore, 

^ T (0) 2 = e ATT r^ 1 T (e)- 1 r^ 1 e AT , (77) 

which is obtained as the matrix inverse of (T7~5l) . Substitution of (1771) into (l59l) yields 

Tt X (Q t ) = Trx^e^T^UTiQ)- 1 ^^) 

= Trx(ry 1/2 e AT Se ATT r- 1/2 ry 1/2 T (e)^ 1 r T 1/2 ) = Tr X (U T (Z)V T (e)), (78) 

where we have used (l48l and Vr from d70l) . By combining ffVVT) with (1681) and d76l) . it follows that 

«9 T p T = rTr(7Je ATT r 7 ; 1 T (e)- 1 r^ 1 e AT ) 

_ * (R s -i r -UT r> p A T T F -ix _ / (r/A)9 T lndetO T (9) for A > 
-rTi(0 T (0) T T e TJe T T ) - - j ^t^q)-^!) f or A = ' (79) 

Here, use is made of the property that, in the case A = 0, the matrix 15 T (Q) from (1701) does not depend 
on T. Integration of the PDE (|79]) yields CZ2]>. Assembling the latter and CZB]) into 08) leads to (|69l). ■ 
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For any constant s > 0, the function St, described by Lemma |3l is invariant under the scaling 
transformation 

x(co) y-¥ U i-> sU, X \-> sX, ri-> sr, (80) 

and so are the ratio A/r and the ODE (l66l) . We now combine Lemmas [2] and [3] to finalise the correction 
scheme (I42"l)-(l45l 

Theorem 4: A nonnegative solution of the covariance HJE (|40l ), satisfying the boundary condition 
(1341 ), is given by 

S T (E,Q) :=lndet(2f/ T (S)) + Tr(f/ T (S) + y r (e) + x(f/r(S)yr(e))). (81) 

Here, 

x (u) = - Vl + 4w - ln(Vl + 4w - 1), (82) 
the map f/y is given by (|48l . and Vr in (|70l) takes the form 

v T (e) = r T 1/2 er T 1/2 . (83) 

Proof: By solving (l66l) as a quadratic equation with respect to ^ follows that 

= (_(Aw + 1) ± ^(Aw + l) 2 - W)/(2w). (84) 

The function x can now be obtained by integrating one of the two regular branches of (|84l) in to. This 
integration is straightforward in the case A := which is shown below to yield a feasible solution in 
(I8TI) . In this case, r must be negative to make real for all u ^ (the trivial situation A = r = 
is excluded from consideration). In view of (|8Q|) . we set r := —1 without loss of generality. Integration 
of = —(1 + a/1 + 4co>) / (2w) yields (f8~2l) . This particular choice of the branch is motivated by the 

identity 

X((v-l)v) = l-2v-ln(2(v-l)) (85) 

for x from (|82l) . which is crucial in what follows to achieve the fulfillment of (1341) . To this end, by 
assembling the starting solution St from Lemma |2] and the trace-analytic correcting function St from 
Lemma [3] (with A := 0, r := -1) into (j42)>. it follows that 

Sr(S,9) =lndetf/ T (S) + Tr[/ T (S) + Trx(^T(S)y T (©)) + p(©) + Try T (6), (86) 

where the map Vr is given by (|70l ) with A = 0, so that 

V T (G) = rr 1/2 U(e) _1 rr 1/2 . (87) 

From (|48l , it follows that if the terminal state covariance matrix is nominally reachable from the 
initial state covariance matrix E in time T, then Z7t(E) = T T 1//2 9r^ 1//2 — 7 n for 9 = Ct(E). Hence, 
S T in d86]) satisfies (O if and only if 

s T (CTXe),e) =indet(er^ 1 -/ n ) + Tr(er T 1 - j n ) 

+ Tr X ((r T 1/2 er r 1/2 - i n )v T (e)) + P (&) + TrV T (e) = o (88) 

for all 6 y T T . With x given by (|82l) . it is possible to find IS, p so as to satisfy (|88l) . Indeed, by setting 
15(6) := _1 and p(0) := nln2, the representation (1871) becomes (l83l) and the boundary value of St 
from (|88l ) takes the form 

^(^(e), 9) = lndet(y T (e) - I n ) + Tr(V T (Q) - I n ) 

+ Tr X ((V T (<?>) - I n )V T (Q)) + n In 2 + W T (0) 
= lndet(2(V T - J n )) + Tr(2Vr - J n ) + Tr X ((V T - I n )V T ). 
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The right-hand side of this equation vanishes for all nominally reachable y T T in view of (|85l) . ■ 
The function St from Theorem @] is a smooth solution of the co variance HJE (l40l) . The minimum 
required supply in the state PDF transition problem (1311) is obtained by combining Theorems [3] and @] 

J T (A/> ,n ),Af(a T ,n T )) =(||«t - M r (a )||p-i + Tr(U T + V T - V % + 4U T V T ) 

- lndet(( ^4 + 4tW T - 4)(2t/ T )- 1 ))/2, (89) 

1/2 1/2 1/2 1/2 

with c/ T = r; v "c T (n )r~ A/ -i n , v T = r~ x/ ^n T r; v ". Here, m t , c t are the nominal state mean and 

covariance semigroups from (TTSl . (fl4l) . and T T is the finite-horizon controllability Gramian from (fl"2l) . 
The structure of the right-hand side of (f8~9l is identical to that in the discrete time case [18], except 
that the semigroups and the gramian are computed in accordance with the continuous time setting. 
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Appendix 

a. differentiation of trace- analytic functions 

The following lemma computes the Frechet derivative for the composition of an analytic function 
with the matrix trace. 

Lemma 4: Let x be a function of a complex variable, analytic in a neighbourhood of E + . Then the 
Frechet derivative of the function P„3Si-> Tr%(S) G C is 

9 s Tr X (S) = X / (S). (Al) 
Proof: Applying an elementary polynomial x( z ) '■= z k of degree k ^ 1 to a matrix S 6 § n yields 
the first variation 5Tr(S fc ) = J^sZl Tr^OE)^- 1 " 5 ) = £;Tr (Steffi); cf. [16, p. 270]. By linearity, 
this proves (IA1I) for arbitrary polynomials x- A standard passage to the limit extends (IA1I) to a power 
series x( z ) '■= ^2t=o c k( z ~ z o) k whose disk of convergence contains the spectrum of S. ■ 
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For the purposes of Section |Xl we complement Lemma |4] by two differentiation formulae, let \& be 
a smooth P n -valued function of the independent time variable T which generates a P n -valued function 
O := of T and S G P„, Then for any X fr° m Lemma H 

(TY X (fi))- = Tr^- 1 + *- 1 i) X '(0)0), (A2) 
dxTr X (n) = y X '(m, (A3) 

where ( ) := &t- Indeed, (IA2I) is obtained by applying the chain rule to the composition of Tr% and 
fiasa function of T for a fixed £ and employing dAD: (Trx(fi)) - = Tr(x'(fi)(^£^ + = 
Tr((\]/\l/ _1 + \& _1 ^/)x'(fi)fi), where the commutativity of fi and x'(^) * s use d. In a similar vein, the 
variation <5Trx(fi) = Tr(x '(fi)#(<5£)#) = Tr(^x'( fi )^ s ) with respect to £ for a fixed T yields 
(IA31 . 

B. Separation of variables for analytic functions of matrices 

The following lemma provides a separation-of-variables technique for analytic functions of matrices 
in Section |X] 

Lemma 5: Let xi> X2 be functions of a complex variable, analytic in a neighbourhood of and real- 
valued on R + . Let G l5 G 2 be § n -valued functions of an independent variable T with TrG^To) ^ for 
some value T of T. Suppose that 

2 

Tr(C7 fc (T)xfc(fi)) = i2(T) for all fi G S+ and T, (Bl) 

fc=i 

where -R(T) is a function of T only. Then Xl , X2 are affinely dependent in their common analyticity 
domain: 

X x + Ax 2 = t, (B2) 

where A and r are real constants. If, in addition to the previous assumptions, xi or X2 is nonconstant, 
then the pair (A, r) is unique, and 

G 2 = XG h R^tTtGl (B3) 
Proof: By considering (|B1I) for scalar matrices fi = win, with u; G R + , it follows that 

2 

^ Xfe(w)TrC7 fe (T) = i2(T) for all u ^ and T. (B4) 

fe=i 

Dividing both sides of dH) for T = T by TrC7i(T ) 7^ and introducing the ratios 

A := TrG 2 (T )/TrG 1 (T ), r := R{T ) /TiG^T,) 

yields the affine dependence (IB2I) of the functions xi an d X2 on R + which extends to their common 
analyticity domain by the identity theorem of complex analysis. Now, let xi or X2 be nonconstant. 
Then (IB2I) determines the pair (A, r) G R 2 uniquely. Therefore, the following implication holds for any 

fi := Gui, fi 2 , fi 3 ) G R 3 : 

fiiXi + H2X2 = A*3 A*2 = A/ii, ^3 = r/ii. (B5) 

For any to ^ and 116K™ with |w| = 1, consider a matrix := couu T . Its eigenvalues are w (with the 
eigenvector u) and (with the eigenspace m -1 , the orthogonal complement of u in R"). The spectral 
decomposition of fi yields x(wuu T ) = x(a;)iiii T + X (0)(I n — uu T ), and (IB 1 1) takes the form 

2 2 
^ Gk{T)u Xk (uj) = R(T) - Xk(0)(TrG k (T) - u T G k (T)u), co ^ 0. (B6) 
fe=i fe=i 
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Here, all u; -independent terms are moved to the right-hand side. By considering (]B6b for fixed but 
otherwise arbitrary values of u and T and recalling (IB5I) . it follows that 



u T G 2 {T)u = Xu T Gi(T)u, (B7) 

2 

R(T) -Y,Xk(0)(TrG k (T) - u T G k (T)u) = ru T G 1 (T)u. (B8) 



k=l 



Since the unit vector u is arbitrary and the matrices G\(T) and G 2 {T) are both symmetric, then (IB7I) 
implies the first of the relations (|B3I) . Combining the latter with (|B2|) gives 

2 

]T Xfc (0)(TrG fc (T) - u T Gk(T)u) = ( Xl (0) + A X2 (0))('IVG 1 (T) - u T Gx(T)u) 

k=l 

= t(TtGi(T) - u T G x {T)u). (B9) 

Substitution of (|B9l ) into (|B8l ) yields 



i2(T) = ™ T Gx(T) M + £ X fc(0)(TrG fe (T) - « T G fc (T) 



fc=i 

= tu^ Gi{T)u + r(TrGi(T) - m t Gi(T)m) = rTrGi(T) 
which, by the arbitrariness of T, establishes the second of the relations (IB3I) . 
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